Transport-Based Neural Style Transfer for Smoke Simulations

Artistically controlling fluids has always been a challenging task. Optimization techniques rely on approximating simulation states towards target velocity or density field configurations, which are often handcrafted by artists to indirectly control smoke dynamics. Patch synthesis techniques transfer image textures or simulation features to a target flow field. However, these are either limited to adding structural patterns or augmenting coarse flows with turbulent structures, and hence cannot capture the full spectrum of different styles and semantically complex structures. In this paper, we propose the first Transport-based Neural Style Transfer (TNST) algorithm for volumetric smoke data. Our method is able to transfer features from natural images to smoke simulations, enabling general content-aware manipulations ranging from simple patterns to intricate motifs. The proposed algorithm is physically inspired, since it computes the density transport from a source input smoke to a desired target configuration. Our transport-based approach allows direct control over the divergence of the stylization velocity field by optimizing incompressible and irrotational potentials that transport smoke towards stylization. Temporal consistency is ensured by transporting and aligning subsequent stylized velocities, and 3D reconstructions are computed by seamlessly merging stylizations from different camera viewpoints.Comment: ACM Transaction on Graphics (SIGGRAPH ASIA 2019), additional materials: http://www.byungsoo.me/project/neural-flow-styl

Deep Fluids: A Generative Network for Parameterized Fluid Simulations

This paper presents a novel generative model to synthesize fluid simulations from a set of reduced parameters. A convolutional neural network is trained on a collection of discrete, parameterizable fluid simulation velocity fields. Due to the capability of deep learning architectures to learn representative features of the data, our generative model is able to accurately approximate the training data set, while providing plausible interpolated in-betweens. The proposed generative model is optimized for fluids by a novel loss function that guarantees divergence-free velocity fields at all times. In addition, we demonstrate that we can handle complex parameterizations in reduced spaces, and advance simulations in time by integrating in the latent space with a second network. Our method models a wide variety of fluid behaviors, thus enabling applications such as fast construction of simulations, interpolation of fluids with different parameters, time re-sampling, latent space simulations, and compression of fluid simulation data. Reconstructed velocity fields are generated up to 700x faster than re-simulating the data with the underlying CPU solver, while achieving compression rates of up to 1300x.Comment: Computer Graphics Forum (Proceedings of EUROGRAPHICS 2019), additional materials: http://www.byungsoo.me/project/deep-fluids

Neural Smoke Stylization with Color Transfer

Artistically controlling fluid simulations requires a large amount of manual work by an artist. The recently presented transportbased neural style transfer approach simplifies workflows as it transfers the style of arbitrary input images onto 3D smoke simulations. However, the method only modifies the shape of the fluid but omits color information. In this work, we therefore extend the previous approach to obtain a complete pipeline for transferring shape and color information onto 2D and 3D smoke simulations with neural networks. Our results demonstrate that our method successfully transfers colored style features consistently in space and time to smoke data for different input textures.Comment: Submitted to Eurographics202

A moving least square reproducing kernel particle method for unified multiphase continuum simulation

In physically based-based animation, pure particle methods are popular due to their simple data structure, easy implementation, and convenient parallelization. As a pure particle-based method and using Galerkin discretization, the Moving Least Square Reproducing Kernel Method (MLSRK) was developed in engineering computation as a general numerical tool for solving PDEs. The basic idea of Moving Least Square (MLS) has also been used in computer graphics to estimate deformation gradient for deformable solids. Based on these previous studies, we propose a multiphase MLSRK framework that animates complex and coupled fluids and solids in a unified manner. Specifically, we use the Cauchy momentum equation and phase field model to uniformly capture the momentum balance and phase evolution/interaction in a multiphase system, and systematically formulate the MLSRK discretization to support general multiphase constitutive models. A series of animation examples are presented to demonstrate the performance of our new multiphase MLSRK framework, including hyperelastic, elastoplastic, viscous, fracturing and multiphase coupling behaviours etc

Lagrangian Neural Style Transfer for Fluids

Artistically controlling the shape, motion and appearance of fluid simulations pose major challenges in visual effects production. In this paper, we present a neural style transfer approach from images to 3D fluids formulated in a Lagrangian viewpoint. Using particles for style transfer has unique benefits compared to grid-based techniques. Attributes are stored on the particles and hence are trivially transported by the particle motion. This intrinsically ensures temporal consistency of the optimized stylized structure and notably improves the resulting quality. Simultaneously, the expensive, recursive alignment of stylization velocity fields of grid approaches is unnecessary, reducing the computation time to less than an hour and rendering neural flow stylization practical in production settings. Moreover, the Lagrangian representation improves artistic control as it allows for multi-fluid stylization and consistent color transfer from images, and the generality of the method enables stylization of smoke and liquids likewise.Comment: ACM Transaction on Graphics (SIGGRAPH 2020), additional materials: http://www.byungsoo.me/project/lnst/index.htm

Incompressible fluid simulation and advanced surface handling with SPH

In den letzten Jahren haben partikelbasierte Methoden zur Simulation von Gasen und Flüssigkeiten in der Computer Graphik an Wichtigkeit gewonnen. Dies da die Repräsentation durch Partikel die Behandlung von freien Oberflächen, Spritzer, Tropfen und komplexen Interaktionen zwischen Objekten erleichtert. Partikelbasierte Methoden weisen jedoch auch Nachteile auf welche das physikalische Verhalten eines Fluids und somit das resultierende visuelle Resultat beeinträchtigen. Obwohl diese Probleme in sozusagen allen partikelbasierten Modellen präsent sind, konzentriert sich diese Dissertation auf die Hauptprobleme der Methode Smoothed Particle Hydrodynamics (SPH). Diese Dissertation beginnt mit einer Einführung in die SPH Methode und erklärt die Schwierigkeit inkompressible Flüssigkeiten zu simulieren. Im Grundmodell von SPH werden Flüssigkeiten durch kompressible Fluide approximiert was zu ungewollten Kompressionsartefakten führt. Obwohl Inkompressibilität erzwungen werden kann, repräsentiert dies den berechenmässig teuersten Teil der Methode, was der Grund ist warum SPH und partikelbasierte Methoden im Allgemeinen weniger geeignet sind um photorealistische Animation von Wasser zu erstellen. In dieser Arbeit präsentieren wir ein neues, inkompressibles Verfahren basierend auf SPH welche Inkompressibilität durch eine Prädiktor-Korrektor Methode erzwingt. Dabei werden die Informationen uber Dichteabweichungen aktiv durch das Fluid propagiert und Druckwerte angepasst, solange bis die Dichtewerte der Partikel einheitlich sind. Mit diesem Ansatz können die Berechnungskosten per Simulationsschritt niedrig gehalten und gleichzeitig ein grosser Simulationszeitschritt verwendet werden. Danach gehen wir auf die Probleme ein welche an den Zwischenflächen von mehreren Fluiden mit unterschiedlicher Dichte entstehen, sowie zwischen Fluiden und festen Objekten. Bei der Simulation von mehreren Fluiden mit dem SPH Grundmodell können Artefakte an der Zwischenfläche beobachtet werden, welche das Verhalten der Fluide negativ beeinflusst. Diese Artefakte sind unphysikalische Oberflächenspannungen sowie numerische Instabilitäten. Diese Dissertation präsentiert ein adaptiertes SPH Modell welches Diskontinuitiäten an den Zwischenflächen von mehreren Fluiden korrekt behandelt und dadurch die Probleme des Grundmodells vermeidet. Des Weiteren prä sentiert diese Arbeit ein einheitliches Modell für die Simulation von Fluiden und festen Objekten um die Interaktion zwischen unterschiedlichen Materialien zu erleichtern. In unserem Modell sind Flüssigkeiten und Gase sowie starre und elastische Körper durch Partikel repräsentiert welche Attribute mit den Objekteigenschaften tragen. Durch das Andern der Attribute können Effekte wie Schmelzen und Erstarren, sowie Vereinigung und Trennung von Objektteilen mit niedrigem Aufwand simuliert werden. Zum Abschluss stellen wir eine neue, effiziente Partikel-Verfeinerungsmethode vor um eine höhere visuelle Qualität beim Rendering von Echtzeit-Flüssigkeiten zu erreichen. Als Ausgangspunkt verwendet unsere Methode die Punktmenge der Simulation und fügt uniform neue Punkte hinzu wobei Oberflächenstrukturen akkurat beibehalten werden. Eine weitere Schwierigkeit von Partikelmethoden ist die Rekonstruktion von glatten Oberflächen. Um dies zu erreichen verwenden wir eine neue Methode, welche den Partikelschwerpunkt der Nachbarschaft bei der Rekonstruktion verwendet, und wir zeigen wie Artefakte in konkaven Regionen erfolgreich vermieden werden können. Particle-based fluid simulations have become popular in computer graphics due to their natural ability to handle free surfaces and interfaces, splashes and droplets, as well as interaction with complex boundaries. However, particle methods have some disadvantageous properties degrading the physical behavior of a simulated fluid and thus the resulting visual quality. Although these problems are present in almost any particle-based fluid solver, this dissertation addresses some of the major problems of the Lagrangian method Smoothed Particle Hydrodynamics (SPH). This thesis starts by reviewing the standard SPH model and its difficulties to satisfy the incompressibility condition. In the standard model, liquids are typically approximated by compressible fluids where pressures are determined by an equation of state, resulting in undesired compression artifacts. Although incompressibility can be enforced, it represents the most expensive part of the whole simulation process and thus renders particle methods less attractive for high quality and photorealistic water animations. In this thesis, we present a novel, incompressible fluid simulation method based on SPH. In our method, incompressibility is enforced by using a prediction-correction scheme to determine the particle pressures. For this, the information about density fluctuations is actively propagated through the fluid and pressure values are updated until the targeted density is sat- isfied. With this approach, the costs per simulation update step can be held low while still being able to use large time steps in the simulation. Next, we shift our attention to the problem of complex interactions between multiple different fluids as well as between fluids and solids. We first focus on the artifacts caused by standard SPH when simulating multiple fluids with high-density ratios. In the standard model, the smoothed quantities of particles near the fluid interface show falsified values and the physical behavior is severely affected, especially if density ratios become large. The artifacts include spurious and unphysical interface tension as well as severe numerical instabilities. In this thesis, we derive a formulation that can handle discontinuities at interfaces of multiple fluids correctly and thus avoids the problems present in standard SPH. With our concepts, an animator has full control over the behavior of multiple interacting fluids. Furthermore, we propose to represent both, fluids and solids, by particles, facilitating the interaction between the different object types. We present a unified simulation model for fluids, rigid, and elastic objects, and show how phase transitions can be modeled by only changing the attribute values of the underlying particles. New effects like merging and splitting due to melting and solidification are demonstrated, and we show that our model is able to handle coarsely sampled and even coplanar particle configurations without further treatment. Finally, we present a novel point refinement method to achieve a higher visual quality of low-resolution fluids. We introduce new algorithms to efficiently upsample an initial point set given by the physical computation. Our method features the ability to accurately preserve surface details and to reach a uniform point distribution. Another challenge is to reconstruct smooth surfaces from the particles. The visualized fluids typically suffer from bumpy surfaces related to the irregular particle distribution. In order to achieve smooth surfaces, this thesis introduces a new surface reconstruction technique based on the center of mass of the particle neighborhood. We show how artifacts in concave regions can be avoided by considering the movement of the center of mass

Temporally Coherent Clustering of Student Data

The extraction of student behavior is an important task in educational data mining. A common approach to detect similar behavior patterns is to cluster sequential data. Standard approaches identify clusters at each time step separately and typically show low performance for data that inherently suffer from noise, resulting in temporally inconsistent clusters. We propose an evolutionary clustering pipeline that can be applied to learning data, aiming at improving cluster stability over multiple training sessions in the presence of noise. Our model selection is designed such that relevant cluster evolution effects can be captured. The pipeline can be used as a black box for any intelligent tutoring system (ITS). We show that our method outperforms previous work regarding clustering performance and stability on synthetic data. Using log data from two ITS, we demonstrate that the proposed pipeline is able to detect interesting student behavior and properties of learning environments